Skip to content

opentelemetry: Implement batch operation for splitting encoded payloads - #299

Open
cosmo0920 wants to merge 1 commit into
masterfrom
cosmo0920-implement-batch-requesting-on-otel
Open

opentelemetry: Implement batch operation for splitting encoded payloads#299
cosmo0920 wants to merge 1 commit into
masterfrom
cosmo0920-implement-batch-requesting-on-otel

Conversation

@cosmo0920

@cosmo0920 cosmo0920 commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add reusable CMetrics APIs for splitting encoded OTLP metric requests
  • cap each generated request by its total OTLP datapoint count
  • preserve resource, scope, metric, and datapoint metadata across batches
  • support gauge, sum, histogram, exponential histogram, and summary metrics
  • cover exact limits, unlimited mode, invalid input, empty contexts, and all metric types

Why

Some OpenTelemetry endpoints, including AWS CloudWatch, reject requests containing
more than 1,000 metric datapoints. CMetrics previously encoded an entire metrics
context into one OTLP request, leaving downstream consumers such as Fluent Bit
to implement their own protobuf splitting logic.

Moving this functionality into CMetrics provides a reusable ownership-safe API
and keeps downstream OpenTelemetry outputs focused on transport and retry
handling.

Validation

ctest --test-dir build -C Debug -R '^cmt-test-opentelemetry$' --output-on-failure
ctest --test-dir build -C Release -R '^cmt-test-opentelemetry$' --output-on-failure
ctest --test-dir build -C Release --output-on-failure (21/21 tests passed)
MSVC AddressSanitizer focused OpenTelemetry test (1/1 test passed)
git diff HEAD^ HEAD --check

The change is additive and does not alter existing encoder entry points or the
OTLP serialized format. A zero datapoint limit preserves the original payload
exactly, while enabled limits produce independently owned requests containing
no more than the configured number of datapoints.

@cosmo0920
cosmo0920 requested a review from edsiper as a code owner August 17, 2026 06:28

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f57271738f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/cmt_encode_opentelemetry_batch.c Outdated
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant